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Abstract 

Online educational videos disseminated content for a university pre-service teacher internship 
program. Placed within an online course management system, the videos were accessed by 202 
interns located in several Western U.S. states. To ascertain the interns’ subjective experiences of 
the online course and videos to assist in the evaluation of the course, the researcher analyzed 
qualitative survey data in the form of constructed response items using a new qualitative-to- 
quantitative protocol. Based on phenomenological and grounded theory methods, this protocol 
was designed to handle the large amount of subjective constructed responses, allowing the 
inductive understanding of the overall experiences of a common phenomenon. The responses 
provided critical information that is useful for instructional designers, online educators, and 
educational video producers. The data suggest: 

1) Universities must carefully consider video hosting options to ensure access. 

2) Online videos should be carefully planned to create high quality, concise videos of less 
than ten minutes in length, yet contain enough educational content to reduce the overall 
number of required videos. 

3) Students appreciate the flexibility online course delivery offered in terms of scheduling 
and eliminating the need to come to campus to attend seminars. 

4) Self-paced online courses require progress indicators to alleviate confusion. 
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Introduction 

Online learning in higher education is outpacing the increasing enrollments in traditional face-to- 
face classes (Allen & Seaman, 2007; 2013), with 21 million postsecondary students in the United 
States enrolled in online courses (Allen & Seaman, 2013). Blended learning, defined and 
structured in a number of ways (Picciano, 2006; Aycock, Garnham, & Kaleta, 2002), capitalizes 
on this growth, in which online and face-to-face components are combined together in the same 
course of study (Norberg, Dziuban, & Moskal, 2011). The potential to reduce didactic lecture time 
within the classroom (Aycock, Garnham, & Kaleta, 2002; Dziuban, Hartman, & Moskal, 2004) in 
favor of interactive teaching and learning gave birth to the “flipped model” of learning where the 
delivery of content occurs in the student’s home (Lage, Platt, & Treglia, 2000; Hamdan, 
McKnight, McKnight, & Arfstrom, 2013; Day & Foley, 2006). While any type of instructional 
material can be utilized in these instructional models, videos are an increasingly popular device 
for content dissemination (New Media Consortium, 2014; Wu & Yamanaka, 2013). Further, 
online and interactive electronic college texts utilizing embedded videos are increasing in usage 
(VitalSource, 2014). With this growth of reliance on online educational videos comes a growing 
need to understand student experiences - positive and negative, so that better online videos can 
be created and utilized. 

The current study adds a valuable tool to the evolving research agenda surrounding online 
educational videos, characterized by “isolated arenas with individual lecturers working on specific 
learning resources” (Shephard, 2003), by suggesting a systematic method of inductively assessing 
common and shared experiences of hundreds of learners to drive continued improvements of this 
medium and mode of transmission. 

A traditional face-to-face seminar training series for pre-service teachers was converted to an 
online course relying on videos that covered the same material - often using the same guest 
speakers. The primary purpose of this article is to present the grounded theory method used to 
assess common and shared experiences of online course and video experiences as expressed by 
the participants of this course, utilizing their voices to drive the inductive categorization process. 

The main research question asked during the process concerned how the learners experienced the 
online course and videos in terms of technical issues, pacing of content, navigation of the course, 
length of videos, etc. Such experience, if properly understood as common and shared experiences 
across the body of over too learners rather than enigmatic personal experiences, would then drive 
updates, alterations, and fixes to future offerings and video productions. Essentially, we needed to 
ascertain what worked, what went wrong, and what we needed to fix. 
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The grounded theory approach chosen allowed the participants to drive the study regarding the 
categorization of their experiences, on their terms, in their words - utilizing a type of data 
gathering technique used within phenomenological research (Polkinghorne, 1989), another type 
of qualitative methodology. This is in contrast to research that relies on a priori hypotheses 
created by the researcher, with control residing in his or her research agenda and personal scope 
of understanding. Grounded theory methods allowed the researcher to “listen to” the experiences 
of the participants as expressed freely without constraining prompts. The result of such methods, 
however, led to the collection of voluminous and nuanced responses from 116 participants who 
submitted 325 unique responses comprising over 12,000 words. This sheer volume of nuanced 
data called for an analysis method that would allow for the interpretation and conversion of 
individual experiences into a high-level understanding of shared and common experiences. A 
mixed qualitative-quantitative method of converting numerous qualitative constructed responses 
into quantifiable data was employed to arrive at higher-level quantitative data, allowing an overall 
measurement of the subjective experiences of hundreds of participants. This method is unique to 
the current body of grounded theory research. 


Literature Review 

Research into the experiences of learners with online videos can draw from classical research into 
the use of classroom-based films that found, for instance, that learners do not like long sessions of 
films presented without significant breaks between showings (Ash, 1950). Learner reactions to 
televised military training films were researched by Kanner, Runyon, and Desiderato (1954) 
utilizing Yes-No and several fill-in items regarding specific aspects of the videos, such as the 
speed of the instructor, whether the learner became bored at the end of the instruction, and the 
desire for more close-up shots. The purpose of that survey was the same as in the current study 
here: to ascertain areas needing improvement for use “in planning succeeding instructional 
presentations” (p. 16), but, unlike the current study, it relied on a priori items to ascertain specific 
attributes of the films. 

Recent investigations into learner experiences with online videos show that learners are 
enthusiastic about online videos (Veeramani & Bradley, 2008; Woo et al., 2008; see Shephard, 
2003 for a summary of case studies from 2002), and prefer videos to academic text (Chan, 2010), 
especially when the former include human elements and a sense of humor (Hibbert, 2014). 
Videos, however, should be integrated into a blended learning system rather than outright 
replacing existing material (Mitra, Lewin-Jones, Barrett, & Williamson, 2010). Learners prefer 
that videos augment, not replace, face-to-face lectures (Cooke et ah, 2012; Dey, Burn, & Gerdes, 
2009) as their main use of videos is for review: preparing for assessments, reviewing topics, and 
so on (Cooke, et ah, 2012; Dey, 2008). While there is a need to determine the “quality of 
experience” in the use of educational videos (Ljubojevic, Vaskovic, Stankovic, & Vaskovic, 2014), 
research is lacking regarding the subjective impacts of online videos as measured by grounded 

This work is licensed under a Creative Commons Attribution 4.0 International License. 


170 



Assessing Experiences with Online Educational Videos: Converting Multiple Constructed Responses to Quantifiable Data 

Harrison 

theory techniques to inductively ascertain learners’ naive descriptions (Polkinghorne, 1989). Such 
techniques allow students to fully explain the breadth of their experiences, without the objective 
constraints placed on their descriptions as is typically found in multiple-choice and Likert-type 
scale items. 

Drawing from phenomenological research methods, the researcher utilized the participants’ 
constructed responses to three items (discussed below), thus gathering what Polkinghorne (1989) 
describes as “the original naive [sic] descriptions of an experience” (p. 47). To analyze the vast 
amount of data collected, the researcher then turned to grounded theory inquiry which also seeks 
to understand the experiences of participants without imposing a priori theories and assumptions 
regarding what the participants may have experienced (Lincoln & Cuba, 1985). Such methods 
were able to thereby generate holistic theories of behaviors, phenomena, experiences, and so on. 
This form of qualitative inquiiy allows for multiple realities as determined by the participants to 
“bubble up” from their experiences, interpretations, and impressions in what is termed as 
naturalistic inquiry (Lincoln & Guba, 1985; Merriam, 2009). The techniques of open coding, 
constant comparison, and axial coding (Glaser & Strauss, 1967) informed the process in the 
examination of the nuanced, detailed, and thematic meanings. The researcher further 
incorporated Charmaz’s (2012) constructivist grounded theory techniques to codify and 
categorize the data, with influence from Thomas’s (2006) discussion of the use of induction to 
inform the process. 

The method of “content analysis” provided a supplemental theoretical paradigm, which, 
according to Krippendorf (2013), “is a research technique for making replicable and valid 
inferences from texts...to the contexts of their use” (p. 24; emphasis original). While content 
analysis emphasizes an examination of extant text (such as determining the authenticity of 
historical texts) rather than hundreds of responses to a survey, it is a useful paradigm in that it 
allows for both quantitative and qualitative methods (Krippendorf [2013, p. 88] criticizes the 
distinction of quantitative and qualitative methods given that all text is qualitative; see Altheide, 
1987). Similarly, Gedik, Kiraz, and Ozden (2012) employed phenomenological methods to 
determine affordances and challenges of blended learning, but involved only ten participants. 

Gedik et al. (2012), Polkinghorne (1989), and Creswell (1998) highlight a shortcoming of the 
above methods: the tendency to focus on fewer participants while producing richer descriptions of 
experiences - fewer but thicker descriptions (Merriam, 2009). The sheer volume of responses 
received in this study deviated from the normal methods of phenomenological and grounded 
theory analysis. In response, the researcher developed an eight-step process he termed 
“Converting Multiple Constructed Responses to Quantitative Data Process,” or CMCR process. It 
is predicated upon the following assumptions, thereby limiting its usefulness while maximizing its 
intended utility: 

1) large number of mutually exclusive participants or cases (at least too), 

2) shared phenomenon or exposure to the same objective events or experiences, 
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3) large amount of response data, 

4) open constructed response items. 

The CMCR process is similar to quantitative content analysis (QCA) in seeking to count the 
number of mentions of terms and ideas within the researched content, “especially the frequency 
and variety of messages, the number of times a certain phrase or speech pattern is used” 
(Merriam, 2009, p. 205). Where the CMCR process differs with QCA, however, is the instrument 
and order of sub-processes: the instrument in the QCA process is the survey instrument, 
documents, transcripts, and so on, whereas the instrument in the CMCR process is the researcher 
who inductively interprets the participants’ constructed responses, writings, and so on; QCA 
begins with the examination of a particular phrase or category and moves to “sampling, data 
collection, data analysis and interpretation” (Altheide, 1987, p. 68), with the consequence of 
forcing narrative data into predefined categories (Altheide, 1987). Conversely, the CMCR process 
begins with the participants, turns to an inductive analysis of the data (open coding for topics, 
creation of categories, and tallying of the categories), and finally to the creation of superordinate 
categories to determine overall participant experiences. This makes the CMCR process inductive 
and naturalistic as opposed to QCA’s deductive and imposed nature. 

Altheide (1987) developed a qualitative counterpart to the quantitative methods that bears noting 
at this point: ethnographic content analysis (ECA). This analysis relies on reflexive qualitative 
coding processes as utilized in grounded theory methods to ascertain “emergent patterns, 
emphases and themes” of ethnographic phenomena (p. 65). It is applicable to a large number of 
participants or texts (Altheide, 1987), making it a good reference for the CMCR process. Yet ECA 
tends to focus on “situations, settings, styles, images, meanings, and nuances” that are 
recognizable by the participants (Krippendorf, 2013, p. 23), rather than the meanings and 
interpretations of experiences of those who create the texts. Further, ECA uses “categories and 
‘variables’ [to] initially guide the study” (Altheide, 1987, p. 68), making it less naturalistic and 
inductive than CMCR. The purpose of the present study was to allow the participants’ responses 
to drive the direction of the categories rather than imposing categories upon the data. 


Methods 

The online course in this study (entitled “College of Education Internship”) addressed 
professional and practical considerations related to the pre-service teacher internship program in 
which the participants were enrolled. This internship program was required by state law to occur 
prior to applying for state licensure. Pre-service teachers, also referred to hereafter as interns, 
were placed into various public school classrooms to initially work with veteran teachers and 
eventually assume full control of the classroom and students. Many aspects of such practical, 
hands-on experience are not covered in the degree program’s curriculum, so are taught to the 
interns during their internship. This instruction traditionally occurred during various mandatory 
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seminar sessions offered throughout each internship period, ranging from eight-hour days on the 
university campus to half-day sessions located in the public schools. Such seminars were 
converted to 30 videos delivered through the university’s online learning management system 
(LMS), Blackboard Learn. Topics covered encompassed working with parent volunteers, teacher 
liability issues, student data confidentiality laws, avoidance of infectious diseases and injuries, 
state school law, applying and interviewing for public school employment, and ethics. Video 
formats included classroom lecture capture featuring both the computer presentation and speaker 
on separate video feeds, narrated computer screen capture, dramatized tutoring situations, and 
live footage of presenters shot indoors and outdoors. Both the researcher and the director of the 
internship program created and edited all videos, but did not possess formal training in the 
particulars of directing, videography, or editing. All 202 interns registered for the internship 
program, at a large Western U.S. university, were automatically enrolled in the online course: 104 
in the Fall 2012 term (20 living more than 50 miles from campus - classified as out-of-area) and 
98 in the following Spring 2013 term (16 out-of-area). Other demographic data were not collected 
in this study. 

A voluntary anonymous survey comprising three constructed response items and a multiple- 
choice item regarding desired video length (detailed below) was provided at the end of the course, 
available through the LMS. For the three constructed response items, 58 interns chose to 
participate in the Fall term and 64 chose to participate in the Spring term, a total of 122 
participants for a 60.4% participation rate (some participants did not respond to all three items). 
For the multiple-choice item regarding desired video length, 145 chose to participate, 68 in the 
Fall term and 77 in the Spring term, for a 71.8% participation rate. 

No benefits were offered for participation in the survey. The participants were allowed to type as 
many words as desired in their response to the first three items (“WebCampus” is the university- 
branded LMS, Blackboard Learn): 

• Issues I had with this WebCampus course are: 

• My positive comments about this WebCampus course are: 

• My suggestions for improvements are: 

A fourth survey item asked the participants to choose their preferred length of videos: “I think the 
length of the videos should be:” with the following multiple-choice responses: 

I- 4 minutes 

5-10 minutes 

II- 14 minutes 

15 - 20 minutes 
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20 minutes or more 

The CMCR process (in Chronological Order) 

The steps below describe the process created, worded in the imperative form for those wishing to 
use or adapt the process. 

1. Table. Place the responses into a four-column table (illustrated below in Figure 1); an 
entire response fills one cell (no division of data units as this would confuse the separation of 
participants). 

2. Initial Topics. Ascertain through open coding the initial topics of the responses (e.g., 
“video access”). No inclusion of value words such as “problematic,” “easy,” “difficult,” and 
“frustrating.” Because participants share the same phenomenon, constantly compare each 
participant’s data to the data from other participants. Make adjustments to earlier categories as 
new data suggest. 

3. Gerund with Value Words. Convert the initial topics into gerund form (_ing) and add 

value words (e.g., “accessing videos problematic”). 

4. Categories. Inductively create categories by comparing the initial topics and gerund forms 
to each other (e.g., “videos inaccessible”). As in Step 2, constantly compare each category to the 
participants’ data and to the other categorical names, modifying wording as the data suggest. 
Then, deductively assess the “fit” of the categories by working in reverse, i.e., could the category 
label “Videos inaccessible” logically include the Initial Topics that currently comprise it? Modify 
category labels and included Initial Topics as needed. 

5. Tally. Tally up the number of times each category was mentioned by the participants 
(referred to as “mentions”); each mention is counted only in terms of how many participants 
mentioned a topic - not in terms of how many times each participant mentioned the topic for that 
survey item. 

6. Repeat. Repeat for each survey item, if applicable. 

7. Record in Spreadsheet. Create a spreadsheet to record the categories and number of 
times each was mentioned, organizing the categories into similar and related groups. 

8. Superordinate Categories. Create superordinate categories encompassing semantically 
and experientially related categories, providing an overall picture of common and shared 
experiences. 

Throughout the process, keep extensive notes on coding decisions, definitions, struggles, and 
procedures. The researcher used a spiral-bound notebook, encompassing about 40 pages of notes. 

CMCR process details. 

Details of the CMCR process are provided below, with an example provided to illustrate the 
process in action (data from the current study). 

1) Response table. Participant responses are copied and pasted into a table comprising four 
columns: “Responses,” “Initial Topics,” “Gerund,” and “Categories,” with the requisite number of 
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rows corresponding to the number of participants. The verbatim text of the responses is placed in 
the “Responses” column, one participant per cell. Data are not divided into data units; instead 
they are left whole per participant so as to not confuse the tallying process to be undertaken in 
Step 5. 

2) Initial topics. Carefully read and initially code each response by identifying the topics suggested 
by the participant’s words (an inductive approach). Use neutral tones, without any judgmental or 
value wording. That is, do not include verbiage such as “worthless,” “good,” “slow,” and “poor.” 
Code each response with the proper number of topics contained therein, such as in Figure 1. Only 
code a topic once - no matter how many times a participant repeats the same topic. Thus, a long 
rant about one issue or a long praise about one favorite aspect holds no more weight than a curt 
description. 

Constantly compare labels to those used by previous and subsequent participants, thus adjust 
“older” labels as necessary, ensuring that the order of responses has negligible influence on the 
final initial topics’ wording. 

Occasionally, a verbatim response may be devoid of contextual meaning, thus rendering the 
response ambiguous. In such instances, pull up the participant’s responses to other survey items 
in search of context meaning (e.g., if a participant stated, “As I said above, it was...” examine the 
participant’s “above” responses from the other survey items to determine the contextual 
meaning). 


Fall 2012 Responses, Item 1 

Issues I had with this WebCampus 
course are: 

Initial Topics: 
topical areas 
discussed - no value 
or judgment 

Gerund: In gerund 
form, with 
positive/negative 
values, e.g.: Pacing 
too slow, Accessing 
videos problematic 
Gerund form at 
beginning 

Categories: 

common, 

determined 

after 

comparison 

I couldn't get the videos to load on 
my old laptop....even [sic] with 
Quicktime. Some of the videos were 
really long and I didn't find much 
value in some of them, although 
others I really enjoyed (I really like 
the male instructor's videos the very 
best....they [sic] were very 
entertaining). 

Videos-access 

Videos-design 

Videos-content 

value 




Figure 1. Response table with verbatim response and initial topics sample. 
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3) Gerund. After completing the initial coding for all participants, translate the initial topics into 

their respective “_ing” forms (the gerund form, see Charmaz, 2012). For example, convert 

“access” to “accessing” and “pace” to “pacing.” Value and judgmental words are added to the 
gerund form, such as including the words “problematic,” “bereft of value,” “slow,” and “easy.” 
Thus, the full meaning of the verbatim response is now seen, such as in Figure 2. 

When participants include information relevant to a different survey item, note such information 
and to which survey item it is transferred (a comment about Prof. XX’s entertaining videos is 
noted in the example as copied to Item 2, as Item 2 addressed participants’ positive experiences). 


Fall 2012 Responses, Item 1 

Issues I had with this 

WebCampus course are: 

Initial Topics: 
topical areas 
discussed - no value 
or judgment 

Gerund: In gerund 
form, with 
positive/negative 
values, e.g.: Pacing 
too slow, Accessing 
videos problematic 
Gerund form at 
beginning 

Categories: 

common, 

determined 

after 

comparison 

I couldn't get the videos to load on 
my old laptop....even with 
Quicktime. Some of the videos 
were really long and I didn't find 
much value in some of them, 
although others I really enjoyed (I 
really like the male instructor's 
videos the very best....they were 
very entertaining). 

[liking Prof. XX’s videos copied to 
Item 2] 

Videos-access 

Videos-design 

Videos-content 

value 

1) Accessing videos 
problematic 

2) Finding some 
videos too long and 
bereft of value 



Figure 2. Response table and gerund form of the initial topics. 


4) Categories. Step away from the coding process for a few days before creating the categories, 
coming back fresh and eager. The categories - located in the far-right column - are created based 
on the topics listed in the Initial Topics and Gerund columns (with occasional comparison to the 
verbatim responses, as needed). Constant comparison within the Categories column is employed 
during the process, similar to Strauss and Corbin’s (1990) axial coding process, in which “data are 
put back together in new ways after open coding by making connections between categories” (p. 
96). Figure 3 shows three categories emerging from the sample data. 


This work is licensed under a Creative Commons Attribution 4.0 International License. 


176 




Assessing Experiences with Online Educational Videos: Converting Multiple Constructed Responses to Quantifiable Data 

Harrison 


Fall 2012 Responses, Item 1 

Issues I had with this WebCampus 
course are: 

Initial Topics: 
topical areas 
discussed - no value 
or judgment 

Gerund: In gerund 
form, with 
positive/negative 
values, e.g.: Pacing 
too slow, Accessing 
videos problematic 
Gerund form at 
beginning 

Categories: 

common, 

determined 

after 

comparison 

I couldn't get the videos to load on 
my old laptop....even with 

Quicktime. Some of the videos were 
really long and I didn't find much 
value in some of them, although 
others I really enjoyed (I really like 
the male instructor's videos the very 
best....they were very entertaining), 
[liking Prof. XX’s videos copied to 
Item 2] 

Videos-access 

Videos-design 

Videos-content 

value 

1) Accessing videos 
problematic 

2) Finding some 
videos too long and 
bereft of value 

Videos 

inaccessible 

Videos 

worthless 

Videos too 
long 


Figure 3. Response table and categorization of the topics. 


The creation of categories involves both inductive and deductive processes. In the inductive 
process, the data from the verbatim Response, Initial Topics, and Gerund columns informs the 
creation of the categories; that is, the categories emerge from the data. The deductive process 
occurs by judging whether some Initial Topics really fit into the Categories by formulating what 
types of ideas and topics would fit each Category, and then “trying out” the Initial Topics to 
determine if they do, indeed, feel like a good fit. For instance, a category called “Accessing the 
course convenient” was first created because several participants indicated their appreciation of 
being able to access the course online, in the evenings after work. As the researcher read 
subsequent participant responses and topics, he found participants who mentioned that they did 
not have to drive to campus to complete the online course. He thought back to the category 
“Accessing the course convenient” and made deductive assumptions as to the type of topics that 
this category could contain. While the category initially referred to a convenience of time 
schedules, he determined that the category could also contain conveniences related to geography 
and travel burdens: each type of convenience was, in fact, a convenience created because the 
course was online. He therefore included the geographic and travel-related convenience within 
this category rather than create a new one specific to geography and travel. 

5) Tally. Once the categories are listed in the Category column, create a simple Tally Table to 
record the categories and the number of mentions each received - organization and order are not 
yet important. 

6) Repeat. Repeat stages 1) through 5) for other survey items, if applicable. 
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7) Record in spreadsheet. Create a spreadsheet to record the tallies from the Tally Table, 
organizing the categories into similar groupings that best encompass the commonalities of related 
categories, as seen in Figure 4. For instance, the categories “Videos inaccessible,” “External links 
broken,” and “Problems accessing WebCampus” related to the same meta-concept of technical 
problems, whereas other categories, such as “Videos worthless” and “Videos too long,” referenced 
non-technical meta-concepts such as the perceived worth of the videos. Such categories were 
organized together in a second group. 


Spring 


Superordinate Fall 2012 2013 

category __ Category _ mentions % mentions % 


Technical 

problems 

common 

Videos inaccessible 





Subtotals 

External links broken 





Assuming WebCampus has inherent 
problems 





Technical issues with quizzes 





Problems uploading to Journals 





Constant upgrading to WebCampus 
confusing 





Problem accessing WebCampus 
















Videos of low 
value 

Subtotals 

Videos worthless 





Too many videos 





Some videos confusing 





Videos too long 











Totals C N=X) 


Figure 4. Recording and organizing categories into a spreadsheet. 


8) Superordinate categories. The grouping process in Stage 7) highlights meta-themes and 
commonalities found between semantically and experientially related categories. This allows for 
the inductive creation of superordinate categories - providing a high-level overview, and a better 
grasp on the overall experience of a common phenomenon. Figure 4 shows two superordinate 
categories, “Technical problems common,” and “Videos of low value.” 
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Results 


The CMCR process revealed 440 initial topics for the survey items, in 78 categories, combined 
into 18 superordinate categories (including Misc), listed in Tables 1 through 3. 

Item 1 asked the participants to discuss problems they experienced. One-hundred-and-sixteen 
participants (Fall N= 58, Spring N=5 8) submitted responses. Twenty-one (18.1%) indicated that 
they experienced no issues or problems. The remaining participants indicated that they 
experienced issues and problems expressed in six superordinate categories, seen in Table 1. 
Technical problems dominated the participants’ experiences (with inaccessible videos a common 
problem), followed by dissatisfaction with the course structure (namely the slow pacing of the 
course for Fall). 

Table 1 

Number of Mentions and Percentages of Categories for Item 1: Issues experienced with 
WebCampus 


Superordinate 


Fall 


Spring 


categories 

Category 

mentions 

96 * 

mentions 

96 * 

Technical 

problems 






common 

Videos inaccessible 

21 

25-3 

8 

11.4 


External links broken 

Assuming WebCampus has inherent 

1 

1.2 

0 

0.0 


problems 

1 

1.2 

0 

0.0 


Technical issues with quizzes 

1 

1.2 

1 

1-4 


Problems uploading to Journals 
Constant upgrading to WebCampus 

3 

3-6 

3 

4-3 


confusing 

0 

0.0 

1 

1-4 


Problem accessing WebCampus 

1 

1.2 

6 

8-5 

Subtotals** 


28 

33-7 

19 

27.1 

Videos of low 
value 

Videos worthless 

6 

7.2 

4 

5-7 


Too many videos 

0 

0.0 

1 

1-4 


Some videos confusing 

0 

0.0 

1 

1-4 


Videos too long 

3 

3-6 

4 

5-7 

Subtotals 


9 

0.1 

10 

14-3 

Course design 
in need of 

Pacing slow 

15 

18.1 

0 

0.0 

improvement 

Forms unavailable 

4 

4-8 

1 

i-4 


Pacing fast 

0 

0.0 

1 

i-4 


Lack of ADA accommodations 

0 

0.0 

1 

i-4 


Navigation difficult 

3 

3-6 

9 

12.9 
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Need progress indicator 

0 

0.0 

9 

12.9 

Subtotals 


22 

26.5 

21 

30.0 

No sense of an 

online 

community 

Deficient feedback from supervisors 

6 

7.2 

0 

0.0 


Lack of online community 

0 

0.0 

1 

t-4 

Subtotals 


6 

7.2 

1 

t-4 

Course of low 

Journal postings worthless 

5 

6.0 

0 

0.0 

value 

compared to 

Too many quizzes 

0 

0.0 

1 

t-4 

actual face-to- 
face teaching 
activities 

Quizzes stressful 

0 

0.0 

1 

t-4 


Course was busywork 

1 

1.2 

6 

8-5 

Subtotals 


6 

7.2 

8 

11.4 

Misc 

Printing manual burdensome 

1 

1.2 

0 

0.0 


Perpetual access after semester is not 
available 

0 

0.0 

1 

t-4 


No issues 

11 

13-3 

10 

14-3 

Subtotals 


12 

14-5 

11 

15-7 

Totals (JV=i53) 


83 

100.0 

70 

100.0 


* Category percentages in all tables are rounded to the nearest tenth of a percent. ** Subtotal percentages in 
all tables are based on superordinate subtotals, not summations of individual category rounded percentages. 


Item 2 asked participants to discuss their positive experiences. One-hundred-and-nine 
participants (Fall N= 45, Spring N= 64) submitted responses. Three (2.8%) indicated that they had 
nothing positive to say. The remaining participants indicated that they were exposed to various 
positive experiences, expressed in five superordinate categories shown in Table 2. Pleasing course 
design and perceived value of the course tied at the top of the list (with efficacious content 
organization and valuable videos driving these superordinate categories), followed by the 
convenience of the online format. 
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Table 2 

Number of Mentions and Percentages of Categories for Item 2: Positive Comments about 
WebCampus Experience 


Superordinate 

category 

Category 

Fall 

mentions 

% 

Spring 

mentions 

% 

Course was 
convenient 
because it was 

online 

Accessing the course convenient 

16 

21.9 

14 

15.6 

Subtotals 


16 

21.9 

14 

15.6 

Technical 

aspects 

worked well 

Submitting assignments to journal easy 

6 

8.2 

0 

0.0 


Links accessible 

0 

0.0 

1 

1.1 


Videos worked fine 

0 

0.0 

2 

2.2 

Subtotals 


6 

8.2 

3 

3-3 

Course design 

was good 

Instructions clear 

4 

5-5 

0 

0.0 


Pacing good 

1 

1-4 

0 

0.0 


Content organization efficacious 

20 

27.4 

28 

311 


Self-paced format valuable 

0 

0.0 

6 

6.7 

Subtotals 


25 

34-2 

34 

37-8 

Course was 

valuable 

Videos valuable 

14 

19.2 

8 

8.9 


Q&A forum valuable 

3 

4-1 

0 

0.0 


Progress monitoring appreciated 
Facilitated interaction with students, 

2 

2-7 

0 

0.0 


supervisors, and UNR personnel 

2 

2-7 

0 

0.0 


Two tries on quizzes good 

1 

1-4 

0 

0.0 


Links valuable 

1 

1-4 

0 

0.0 


User control of videos appreciated 

0 

0.0 

1 

1.1 


Assignment checklist valuable 

0 

0.0 

1 

1.1 


Quizzes valuable 

0 

0.0 

2 

2.2 


Course valuable 

2 

2.7 

13 

14.4 


Journals valuable 

0 

0.0 

11 

12.2 

Subtotals 


25 

34-2 

36 

40.0 


None 

1 

1-4 

2 

2.2 

Misc 

Good help support 

0 

0.0 

1 

1.1 

Subtotals 


1 

14 

3 

3-3 

Totals 

(JV=i6 3 ) 


73 

100.0 

90 

100.0 
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Item 3 asked participants to suggest improvements. One-hundred participants (Fall N= 39, Spring 
N=6i) submitted responses. The participants suggested a variety of improvements as expressed in 
seven superordinate categories, seen in Table 3. The most common suggestion was to improve the 
course design (comprised mainly of the suggestion to open all modules at the beginning, that is, 
to allow for self-pacing), followed by the suggestion to improve the quality of the videos while 
making fewer of them. These suggestions were taken into consideration between semesters by 
opening all modules at the beginning for the Spring 2013 semester and condensing several videos 
(10% of the total video offerings). 

It is interesting to note that after the change to a self-paced format, the Spring participants 
indicated that they were confused as to where they should be at what point, with the desire for a 
progress indicator, seen in Table 1 under “Course design in need of improvement,” and Table 3 
under “Reconsider the current course design...” and “Provide a progress indicator...” While 
qualitative research does not seek to establish causality, logic would suggest that the conversion of 
the online course into a self-paced format likely brought about such confusion and need for 
progress indication. 

Table 3 

Number of Mentions and Percentages of Categories for Item 3: Suggestions to Improve the 
Internship Orientation Experience 


Superordinate 

category 

Category 

Fall 

mentions 

% 

Spring 

mentions 

% 

Create higher 

Reduce number of videos 

3 

5-4 

0 

0.0 

quality videos 
that are fewer 
in number and 

Provide PPT slides (used in videos) or 
study guide for review 

2 

3-6 

0 

0.0 

more concise 

Provide a video length indicator 

1 

1.8 

0 

0.0 


Make videos less frightening (tone 
down the warnings) 

1 

1.8 

0 

0.0 


Utilize written format in place of some 
videos 

0 

0.0 

1 

t-4 


Create concise videos 

7 

12.5 

10 

14-3 


Create more encompassing videos, but 
fewer of them 

0 

0.0 

2 

2.9 


Create better quality videos 

1 

1.8 

11 

15-7 

Subtotals 


15 

26.8 

24 

34-3 

Reconsider the 
current course 
design and 
layout to 
improve ease 
of use and 
navigation 

Open all modules at the beginning 

14 

25.0 

0 

0.0 


Clarify directions 

1 

1.8 

0 

0.0 


Provide opportunities for student-to- 
student interaction 

4 

7-t 

4 

5-7 
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Customize layout for dual-major 
interns 

0 

0.0 

1 

1-4 


Create adaptive format - if they know 
it, they can skip it 

0 

0.0 

1 

1-4 


Provide some face-to-face in place of 
some online components 

0 

0.0 

1 

1-4 


Group documents together in one 
location 

2 

3-6 

4 

5-7 


Combine all content sources into one 
location 

0 

0.0 

2 

2.9 


Reorganize course layout 

0 

0.0 

7 

10.0 

Subtotals 


21 

37-5 

20 

28.6 

Provide a 
progress 
indicator to 
alleviate 
confusion 

Provide a progress indicator 

3 

5-4 

8 

11.4 

Subtotals 


3 

5-4 

8 

11.4 

Reduce the 
required work 

Reduce number of Journal 
assignments 

7 

12.5 

3 

4-3 

load of course 

Reduce number of quizzes 

l 

1.8 

1 

1-4 

to concentrate 
on teaching 

Reduce online course load 

0 

0.0 

4 

5-7 

Subtotals 


8 

14.2 

8 

11.4 

Address 

technical issues 

Improve video access 

3 

5-4 

1 

1-4 


Ensure Web links are good 

2 

3-6 

0 

0.0 


Make compatible with iOS devices 

0 

0.0 

1 

i-4 

Subtotals 


5 

8.9 

2 

2.9 

Conduct 

personnel 

Supervisors should provide feedback 
on Journals 

3 

5-4 

0 

0.0 

training 

workshops 

Train supervisors how to use 

WebCampus 

l 

1.8 

0 

0.0 


Enhance communication between 
internship personnel 

0 

0.0 

3 

4-3 

Subtotals 


4 

7-1 

3 

4-3 

Misc 

Remind students to compose in Word, 
then copy-paste to Journal 

0 

0.0 

1 

1-4 


Update content 

0 

0.0 

1 

1-4 


Allign test instruments and video 
content 

0 

0.0 

1 

1-4 


Allow lead teachers to view course 

0 

0.0 

2 

2.9 

Subtotals 


0 

0.0 

5 

7-1 

Totals (JV=i26) 


56 

100.0 

70 

100.0 
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In the multiple-choice survey item regarding participants’ preferred length of videos, of the 145 
participants (Fall N= 68, Spring N =-/-/) 53.8% preferred videos to be between five and ten minutes 
in length, and an additional 19.3% preferred shorter videos, seen in Table 4. Overall, 73.1% of 
participants preferred educational videos to be under ten minutes in length. 

Table 4 

Preferred Length of Videos by Number of Participants 


Length 

Fall 

% 

Spring 

% 

Overall 

number 

Overall % 

1-4 minutes 

13 

19.1 

15 

19-5 

28 

19-3 

5-10 minutes 

33 

48.5 

45 

58.4 

78 

53-8 

11-14 minutes 

11 

16.2 

10 

13.0 

21 

14-5 

15 - 20 minutes 

9 

13.2 

6 

7.8 

15 

10.3 

20 minutes or 

2 

2.9 

1 

1-3 

3 


more 




2.1 

Unanswered 

0 

0.00 

0 

0.0 

0 

0.00 


Discussion and Conclusion 

The increasing reliance on online education and videos within all levels of education calls for an 
increasing understanding of the experiences of learners in order to ascertain areas in need of 
improvement and adjustment, from technical to artistic issues. Given the vast number of learners, 
breadth of experiences, and complex nuances of language used by learners to describe their 
experiences, a method to, in essence, distill the common shared experiences of learners is needed. 
Such a method is best situated within phenomenological and grounded theory inquiries that 
allows the learners to drive the categorization of their experiences, rather than relying on the a 
priori hypotheses and research questions crafted by researchers who must assume what types of 
experiences online learners may have, and then measure accordingly. 

This study collected experiential data from 116 participants who shared their myriad of positive 
and negative experiences using over 12,000 words. An inductive coding process drawing from the 
constructivist grounded theory methods proposed by Charmaz (2012) and ethnographic content 
analysis by Altheide (1987) was created: Converting Multiple Constructed Response Items to 
Quantitative Data (CMCR). This eight-step process revealed 18 superordinate categories of 
experience, comprising 78 “sub” categories. Many of the issues (both positive and negative) were 
of surprise to the researcher and the seminar director, driving improvements to the course and 
videos. 

While we expected that the participants would appreciate the flexibility offered by the online 
course and videos - in terms of scheduling and coming to campus, we did not anticipate other 
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findings. Thus, a study that relied on a priori categories may have missed such important 
common and shared experiences. The main findings were: 

First, many participants noted a serious issue regarding technical access to the videos. The issues 
ranged from audio-video synching and truncated playback, to the complete inability to access 
some videos. Much of the problem centered on the university’s video hosting system. It was 
designed to serve up videos without any codec or bit rate conversion - that is, the exact file that 
was placed on it was the exact file that was streamed to the participants. For instance, we created 
large video files utilizing the .MOV shell with the H.264 codec inside, which is exactly the same 
file as was served up to the participants - no matter their devices, video viewers, or Internet 
connection speeds. This is the opposite operation in use by popular video hosting services such as 
YouTube and Vimeo, in that they convert uploaded videos into the proper codec and bit rate as to 
be most compatible with the end user’s device and Internet connection speed. The feedback 
received in this study influenced the university to purchase a corporate video hosting solution that 
offered such conversions (it went online in late 2013). 

Second, the participants desired concise videos that also displayed better production quality. 
Many of the videos were over 20 minutes in length, whereas almost three-fourths of the 
participants indicated that they preferred the videos be less than ten minutes. One limitation to 
this survey item is that it made no differentiation between video content and artistic format (e.g., 
documentary, talking-head, dramatized), nor between situational contexts where the participants 
watched the videos (e.g., in their living room, at a coffee shop, at work during a break, on a bus). 
Such factors may influence learner preferences and should be taken into account in future 
research. A second limiting factor to these findings concerns why the participants prefer shorter 
videos (e.g., they can only consume short videos while on a work break, possess short attention 
spans, tire easily when watching “boring” material, have a lack of motivation). Qualitative inquiry 
would likely help elucidate such reasons and provide suggested prescriptive video design. This 
vein of research holds implications for the field of distance education, discussed below. 

We learned that video production is a much more involved process than we anticipated and that 
proper planning, production, and hosting of videos require more demanding technical skills and 
pedagogical understanding than we then possessed. 

Third, while one of the purposes of the conversion was to eliminate the burden of face-to-face 
seminars endured by the internship staff and director, the participants also greatly appreciated 
the flexibility in scheduling content consumption around their busy lives, while not having to 
come to campus. 

Fourth, self-paced online courses, where there is a minimal amount of continuous student- 
instructor contact (as was the case in this course), can be confusing for students, requiring some 
type of progress indicator to help them know what to do when, in what order, and what they still 
have yet to do. 
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The process created for this study, the CMCR process, may be a valuable tool for those 
researchers looking to collect the subjective, freely given responses from hundreds of participants 
who experienced the same phenomenon, to inductively ascertain common, shared experiences 
and interpretations thereof. Such a process helps attain a high-level understanding that is difficult 
to grasp given the many nuances, phrasing, and terminology such a vast data collection process 
will produce. Such a technique is applicable for a broad range of applications, including corporate 
education, online K-12 schools, and health care education. 


Limitations 

Limitations of this study include the lack of demographic data collected for participants, including 
computer abilities and self-efficacy data. While this study employed the Technology and Internet 
Assessment (TIA) survey instrument (Ealy, 2000) to ascertain the latter data, the researcher was 
not able to connect participant data from the TIA survey to those data collected in the 
experiential, constructed response survey. Examination of the 45 TIA survey participants (22.2% 
participation rate) revealed above-average computer and Internet abilities and self-efficacy 
perceptions compared to nationally normed data. It is only possible, however, to make speculative 
inference that the TIA data may be representative of those who participated in the experiential 
survey, thus such data are not reported above. 

Other limits include the context in which these data were collected: an authentic learning 
environment encompassing multiple locations around the Western United States. Such 
authenticity enhances external validity, but makes it more difficult for the researcher to account 
for confounding variables, such as type of computer equipment used by the participants, Internet 
service provider speed and reliability, time-of-day impacts, home environment variables, and so 
on. Learner control of multimedia can increase learning outcomes (Mayer & Chandler, 2001), but 
such data were not available through the video hosting system in use during the study. Future 
research should look at collecting more specific data regarding such confounding variables, as 
well as learner control of videos: pausing, rewinding, and re-watching behaviors. Such a study, of 
course, would employ quantitative methodologies rather than qualitative methodologies as used 
in the present study. 

The current study exhibited several video design and access concerns that were unique to the 
institution. Thus, these findings may vary if a different video-hosting service was used. The 
impetus for this study was to ascertain such issues, many of which were corrected for future 
iterations. 
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Implications 

Three main implications for the field of distributed education research emerged from this study: 
the production quality of videos may directly and indirectly impact learning outcomes, preferred 
length of videos may impact learning outcomes, and there may be optimal or preferred content 
areas to be presented with the video medium. 

Direct impacts of video production quality may result because of interference in the message 
causing confusion or incomprehensibility, distortion, or inaudible sound/non-viewable visuals. 
Indirect impacts include the possibility that low quality compared to expected quality may lower 
viewer interest and attention, or subconsciously devalue the information presented in the video 
(e.g., if the video was not worth spending considerable effort to create, the content must not be 
worth much, either). Further, incomprehensibility of content lowers viewer attention (Anderson, 
Lorch, Field, & Sanders, 1981), thus likely leading to decreased encoding of content. 

The preferred length of videos may fall under a discussion of viewer attention spans. While a full 
discussion of the field of vigilance decrement theory (Parasuraman, 1986; Johnstone & Percival, 
1976; Smith, 2006) and its applicability to pedagogical design (Young, Robinson & Alberts, 2009) 
is outside the scope of this paper, it is certainly a concept in need of fuller study. Classical 
research conducted during World War II regarding the impact of massed presentations (longer 
sessions) versus spaced presentations (sessions separated by breaks) offers guidance (e.g., Hoban, 
1946; Ash, 1950), however, the educational contexts studied were very different than today’s 
online situations that afford private viewing and learner control. More recent research into the 
concept of segmentation of multimedia modules as found in Mayer’s work (e.g., Mayer & 
Chandler, 2001; Mayer, Dow, & Mayer, 2003) are also informative, bearing in mind that 
segmentation in this sense is limited to videos lasting only seconds - impractical lengths in real- 
world educational settings. Ibrahim, Antonenko, Greenwood, and Wheeler (2012) offer support in 
favor of videos about six minutes in length versus 32 minutes, but confounding variables 
obscured the clarity of the treatments on the study’s results. Thus the researcher strongly believes 
that further clarification of the impacts of video length - to include comparisons of massed versus 
spaced videos - be re-examined in light of online, private, and learner-controlled access factors. 

Finally, it is also suggested that, based upon some participants’ comments, future research 
examine the type of video content preferred by learners in terms of learning effectiveness. 
Research regarding content areas involving process or procedural content versus factual or 
abstract knowledge should be conducted - as some content may best be left to text. Classical 
media comparison studies examined this topic in terms of comparing film to other forms of 
delivery modes (e.g., Wood & Freeman, 1929; Wise, 1939; Hall & Cushing, 1947; Kanner, Runyon, 
& Desiderato, 1954), offering starting points for future research. 
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